Dual camera module including hyperspectral camera module, apparatuses including dual camera module, and method of operating the same

ABSTRACT

A dual camera module including a hyperspectral camera module, an apparatus including the same, and a method of operating the apparatus are provided. The dual camera module includes a hyperspectral camera module configured to provide a hyperspectral image of a subject; and an RGB camera module configured to provide an image of the subject, and obtain an RGB correction value applied to correction of the hyperspectral image.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.17/105,103, filed Nov. 25, 2020, which claims priority to Korean PatentApplication No. 10-2019-0160962, filed on Dec. 5, 2019, in the KoreanIntellectual Property Office, the disclosure of which is incorporatedherein in its entirety by reference.

BACKGROUND 1. Field

Example embodiments consistent with the present disclosure relate tocamera modules and applications thereof, and more particularly, to dualcamera modules including a hyperspectral camera module, apparatusesincluding the same, and methods of operating the apparatuses.

2. Description of Related Art

A hyperspectral image may be measured by a scanning method or a snapshotmethod. The scanning method may be realized by combining scanningequipment with a spectral image sensor and may simultaneously acquire animage and a spectrum by exchanging a slit-like spectrum or a frontfilter. The snapshot method is a non-scanning method of measuring ahyperspectral image by implementing different filters directly on animage pixel.

SUMMARY

Provided are dual camera modules that increase resolution of ahyperspectral image.

Provided are dual camera modules including a hyperspectral camera modulethat may correctly recognize a shape of a three-dimensional subject.

Provided are dual camera modules including a hyperspectral camera modulethat may correct an error due to a distance between cameras.

Provided are hyperspectral camera modules that may increase resolutionof a hyperspectral image.

Provided are apparatuses including a camera module that may increaseresolution of a hyperspectral image.

Provided are methods of operating the apparatuses.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments of the disclosure.

According to an aspect of an example embodiment, there is provided adual camera module providing a corrected image of a hyperspectral image,the dual camera module including: a hyperspectral camera moduleconfigured to provide a hyperspectral image of a subject; and an RGBcamera module configured to provide an image of the subject, and obtainan RGB correction value applied to correction of the hyperspectralimage.

The image of the subject provided by the RGB camera module may includean RGB image obtained by using red light, green light, and blue light asa main light, or an image obtained by using infrared light together withred light, green light, and blue light as the main light.

The main light may further include ultraviolet light.

The RGB camera module may include a first optical guide module and afirst image sensor configured to sense the image of the subject.

The first image sensor may include a plurality of pixels, and each pixelof the plurality of pixels may include four different sub-pixels.

The first image sensor may include a plurality of pixels, and each pixelof the plurality of pixels may include six sub-pixels.

Five sub-pixels of the six sub-pixels may be different from each other.

The hyperspectral camera module may include: a second optical guidemodule; a hyperspectral filter configured to generate the hyperspectralimage; and a second image sensor configured to sense the hyperspectralimage.

The dual camera module may further include a low pass filter providedbetween the second optical guide module and the hyperspectral filter.

According to an aspect of an example embodiment, there is provided anelectronic apparatus including: a display on which an image of a subjectis displayed; a dual camera module configured to provide a correctedimage of a hyperspectral image of the subject; a driving control circuitunit configured to control the dual camera module; and a battery thatprovides power to the dual camera module, wherein the dual camera moduleincludes a hyperspectral camera module configured to provide thehyperspectral image of the subject; and an RGB camera module configuredto provide the image of the subject and obtain an RGB correction valueapplied to correction of the hyperspectral image.

The electronic apparatus may further include a light source configuredto illuminate the subject.

According to an aspect of an example embodiment, there is provided amirror-type display apparatus, including: a mirror-type display; a lightsource provided around the mirror-type display; a first camera modulethat provides a first image of a subject; and a second camera modulethat provides a second image of the subject, wherein the first imagehave first spectral characteristics and the second image have secondspectral characteristics which are different from the second spectralcharacteristics.

One of the first camera module and the second camera module may includea hyperspectral camera module.

According to an aspect of an example embodiment, there is provided amethod of operating an apparatus including a dual camera module, themethod including: acquiring, by a first camera module, a first imageincluding first spectral information; acquiring, by a second cameramodule different from the first camera module, a second image includingsecond spectral information, an amount of the second spectralinformation being greater than an amount of the first spectralinformation; and increasing a resolution of the second image by usingthe first spectral information of the first image.

The first image may be an RGB image having a resolution that is higherthan the resolution of the second image.

The second image may be a hyperspectral image.

The increasing of the resolution of the second image may include:obtaining a first average value and a deviation of the first spectralinformation of the first image; obtaining a second average value of thesecond spectral information of the second image; and adding thedeviation to the second average value.

The obtaining of the first average value and the deviation may include:obtaining a first average R value, a first average G value, and a firstaverage B value of a plurality of pixels included in a first imagesensor by which the first image is sensed; and obtaining, for each pixelof the plurality of pixels, deviations between the first average Rvalue, the first average G value, and the first average B value and arespective R value, G value, and B value of the pixel.

The obtaining of the second average value may include: dividing a unitpixel of a second image sensor, by which the second image is sensed,into a plurality of spectral pixels, wherein each spectral pixel fromamong the plurality of spectral pixels includes a plurality of channels;and obtaining a second average R value, a second average G value, and asecond average B value for the plurality of spectral pixels, wherein anumber of the plurality of spectral pixels is less than a number of theplurality of channels, and the number of the plurality of spectralpixels varies depending on a number of sub-pixels included in each pixelfrom among the plurality of pixels included in the first image sensor.

The number of sub-pixels may be four or six.

According to an aspect of an example embodiment, there is provided ahyperspectral camera module providing a corrected hyperspectral image,the hyperspectral camera module including: a first camera moduleconfigured to provide a first image of a subject; and a second cameramodule configured to provide a second image of the subject, wherein thesecond image is different from the first image.

The first image may include an RGB image.

The first image may include an image obtained by using red light, greenlight, blue light, and infrared light as main light.

The first image may include an image obtained by using red light, greenlight, blue light, and UV light as main light.

The first image may include an image obtained by using red light, greenlight, blue light, infrared light, and UV light as main light.

The hyperspectral camera module may be configured to obtain an RGBcorrection value based on the first image and to obtain a hyperspectralimage based on the RGB correction value.

The second image may include an uncorrected hyperspectral image.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certain exampleembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a cross-sectional view of a dual camera module including ahyperspectral camera module according to an example embodiment;

FIG. 2 is a cross-sectional view of a dual camera module including ahyperspectral camera module according to another example embodiment;

FIG. 3 is a cross-sectional view of an example of an optical guidemodule of the hyperspectral camera modules of FIGS. 1 and 2 ;

FIG. 4 is a plan view of a pixel distribution of an image sensor of anRGB camera modules of FIGS. 1 and 2 ;

FIG. 5 is a plan view of an example of a hyperspectral filter of FIG. 3;

FIG. 6 is a plan view showing a correspondence relationship betweenchannels included in a unit pixel of the hyperspectral filter of FIG. 5and virtual spectral pixels;

FIG. 7 is a graph showing an example of RGB spectral characteristics offirst to fourth pixels P1 to P4 of the RGB camera module of FIG. 4 ;

FIG. 8 is a graph showing an example of a spectrum obtained throughchannels included in a unit pixel of the hyperspectral filter of FIG. 5;

FIG. 9 is a graph showing a deviation range of R, G, and B of fourvirtual spectral pixels corresponding to unit pixels of thehyperspectral filter of FIG. 6 , and RGB average values and deviationsof four pixels of an RGB camera module representing RGB spectralcharacteristics of FIG. 7 ;

FIG. 10 is a plan view showing a case in which pixels of the RGB cameramodules of FIGS. 1 and 2 are configured with four different sub-pixelsR, G, B, and IR;

FIG. 11 is a plan view showing a case in which pixels of the RGB cameramodules of FIGS. 1 and 2 are configured with four different sub-pixelsR, G, B, and UV;

FIG. 12 is a plan view showing a case in which pixels of the RGB cameramodules of FIGS. 1 and 2 are configured with six sub-pixels UV, R, G1,IR, G2, and B;

FIG. 13 is a plan view showing a plurality of virtual spectral pixelscorresponding to N channels, as an example of a unit pixel of thehyperspectral filter of FIG. 3 , when the unit pixel includes the Nchannels and a unit pixel of an RGB camera is configured with foursub-pixels;

FIG. 14 is a plan view showing a plurality of virtual spectral pixelscorresponding to N channels, as an example of a unit pixel of thehyperspectral filter of FIG. 3 , when the unit pixel includes the Nchannels and a unit pixel of an RGB camera is configured with sixsub-pixels;

FIG. 15 is a perspective view of a unit pixel of the hyperspectralfilter of FIG. 5 ;

FIG. 16 is a cross-sectional view of a first channel of a unit pixel ofFIG. 15 ;

FIG. 17 is a hyperspectral photograph of a hand taken by using thehyperspectral camera module of the dual camera module of FIG. 1 or FIG.2 ;

FIG. 18 is a graph showing a light absorption spectrum of a hand due tohemoglobin and melanin, which is obtained from the hyperspectralphotograph of FIG. 17 , superimposed with a reference spectrum;

FIG. 19 is a photograph of a hand photographed using the RGB cameramodule of FIG. 1 or 2 .

FIG. 20 is a hyperspectral photograph photographed using thehyperspectral camera module of FIG. 1 or 2 and shows spectralcharacteristics (light absorption characteristics) of a hand due tohemoglobin;

FIG. 21 is a photograph to enlarge the size of the hyperspectral imageof FIG. 20 to the RGB image level of FIG. 19 ;

FIG. 22 shows a corrected hyperspectral image obtained as a result ofapplying information obtained from the spectral characteristics of theimage photographed by using the RGB camera module of FIG. 1 or 2 to thehyperspectral image of FIG. 21 ;

FIG. 23 is a plan view of an RGB correction for each of the spectralpixels corresponding to unit pixels of the hyperspectral filter;

FIG. 24 is a plan view of a case in which a subject is photographed byusing an apparatus on which a dual camera module according to an exampleembodiment is mounted;

FIG. 25 shows an image (a) of a subject showing the effect of unevennessof illumination and an image (b) of the subject when the unevenness ofillumination is corrected in a subject photographing by using theapparatus of FIG. 24 ;

FIG. 26 is a perspective view of a mobile phone as one of electronicapparatuses including a dual camera module according to an exampleembodiment;

FIG. 27 is a side view of a mirror-type display apparatus including adual camera module according to an example embodiment;

FIG. 28 is a front view of a mirror-type display apparatus including adual camera module according to an example embodiment; and

FIG. 29 is a flowchart of a method of operating an electronic apparatusincluding a dual camera module, according to an example embodiment.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings, wherein like referencenumerals refer to like elements throughout. In this regard, exampleembodiments may have different forms and should not be construed asbeing limited to the descriptions set forth herein. Accordingly, theexample embodiments are merely described below, by referring to thefigures, to explain aspects.

As used herein, the term “and/or” includes any and all combinations ofone or more of the associated listed items. Expressions such as “atleast one of,” when preceding a list of elements, modify the entire listof elements and do not modify the individual elements of the list. Forexample, the expression, “at least one of a, b, and c,” should beunderstood as including only a, only b, only c, both a and b, both a andc, both b and c, or all of a, b, and c.

The advantages, features, and methods of achieving the advantages may beclear when referring to the example embodiments described below togetherwith the drawings. However, embodiments may have different forms andshould not be construed as being limited to the descriptions set forthherein. Rather, these example embodiments are provided so that thisdisclosure will be thorough and complete, and will fully convey thescope of the disclosure to those of ordinary skill in the art.Embodiments will be defined by the appended claims. In the drawings,thicknesses of layers and regions may be exaggerated for convenience ofexplanation.

Terminologies used in the specification will be briefly described andthe current example embodiment will be described in detail.

Terminologies used herein are selected as commonly used by those ofordinary skill in the art in consideration of functions of the currentexample embodiment, but may vary according to the technical intention,precedents, or a disclosure of a new technology. However, the terms mayhave different meanings according to the intention of one of ordinaryskill in the art, and in this case, the meaning of the selected termswill be described in detail in the detailed description of thedisclosure. Thus, the terms used herein should be defined based on themeaning of the terms together with the description throughout thespecification.

It should be understood that, when a part “comprises” or “includes” anelement in the specification, unless otherwise defined, it is notexcluding other elements but may further include other elements.

Hereinafter, a dual camera module including a hyperspectral cameramodule according to example embodiments, apparatuses including the same,and a method of operating the apparatuses will be described in detailwith reference to the accompanying drawings. Since the dual cameramodule described below, as a result, will provide a hyperspectral image,it may also be referred to as a hyperspectral camera module. Embodimentsdescribed below are merely examples, and various modifications arepossible. It will be understood that when an element or layer isreferred to as being “on” or “above” another element or layer, theelement or layer may be directly on another element or layer or theremay be intervening elements or layers.

FIG. 1 is a cross-sectional view of a dual camera module 100 including ahyperspectral camera module according to an example embodiment.

Referring to FIG. 1 , the dual camera module 100 including ahyperspectral camera includes two different types of camera modules,that is, first and second camera modules 110 and 120 and a first lightsource 130. The first and second camera modules 110 and 120 may havedifferent configurations and uses from each other. The first cameramodule 110 is a visible light camera module for acquiring a first image,that is, an RGB image of a subject 150 by using red light R, green lightG, and blue light B as main light. Therefore, the first camera module110 may be referred to as an RGB camera module 110. The first cameramodule 110 may be a general camera module, not a hyperspectral camera.For example, the first camera module 110 may be a camera module mountedon a small mobile device, such as a mobile phone for taking orphotographing an image of the subject 150 by using R, G, and B as mainlight. The first camera module 110 may include a first optical guidemodule 110A and a first image sensor 110B, wherein the first opticalguide module 110A receives light L2 reflected from the subject 150 amonglight L1 emitted from the first light source 130 to the subject 150 andallows the light L2 to reach the first image sensor 110B. The firstoptical guide module 110A may include a lens or a lens system and may bedisposed in front of a surface of the first image sensor 110B on whichthe light L2 is incident. The light L2 reflected from the subject 150passes through the first optical guide module 110A and enters the firstimage sensor 110B. The first optical guide module 110A is between thesubject 150 and the first image sensor 110B. The first optical guidemodule 110A may include a single lens or two or more lenses on theoptical axis. The first image sensor 110B provided to sense the firstimage transmitted from the first optical guide module 110A may be, forexample, a CMOS image sensor, but is not limited thereto. The firstlight source 130 may be a light source providing a sufficient amount oflight or a minimum amount of light necessary for capturing an image ofthe subject 150 by using the first camera module 110 in an environmentin which there is not enough illumination to capture an image of thesubject 150. If external light provides illuminance sufficient tocapture an image of the subject 150, the first light source 130 may notbe used. In one example, the first light source 130 may or may not beincluded in the dual camera module 100 as an optional member. The firstlight source 130 may be used as a general lighting means by beingconnected to a power source of an apparatus on which the dual cameramodule 100 is mounted regardless of the use of the dual camera module100. The second camera module 120 disposed parallel to the first cameramodule 110 may be a camera module having a configuration and purposedifferent from that of the first camera module 110. For example, thesecond camera module 120 may be a hyperspectral camera module 120 thatprovides a hyperspectral image of the subject 150. The second cameramodule 120 may include a second optical guide module 120A and a secondimage sensor 120B. Since the use of the second camera module 120 iscompletely different from that of the first camera module 110, theconfiguration of the second optical guide module 120A may be differentfrom that of the first optical guide module 110A. The subject 150 may bea person or an object. If subject 150 is a human, the subject 150 may bea hand, face, or skin of a specific part.

A hyperspectral image of the subject 150 may be recorded in the secondimage sensor 120B. The hyperspectral image may include spectralinformation (i.e., a spectrum) together with image information about thesubject 150. The second image sensor 120B may be a CMOS image sensor.

When the dual camera module 100 of FIG. 1 is used, an RGB image of thesubject 150 may be obtained through the first camera module 110, and atthe same time, a hyperspectral image of the subject 150 may be obtainedthrough the second camera module 120. The resolution of thehyperspectral image obtained through the second camera module 120 may beimproved by using the spectral information of the RGB image obtainedthrough the first camera module 110, which will be described later. Thefirst and second camera modules 110 and 120 may be operated in realtime. Therefore, when the dual camera module 100 is mounted in anapparatus including a communication module and a display module (forexample, a medical display apparatus or a mobile phone), an RGB imageand a hyperspectral image of the subject 150 may be observed in realtime, and the correction with respect to the hyperspectral image may beperformed in real time. In other words, the resolution improvement ofthe hyperspectral image may be observed in real time.

FIG. 2 is a cross-sectional view of a dual camera module 200 including ahyperspectral camera module according to another example embodiment.Only descriptions different from the dual camera module 100 of FIG. 1will be given.

Referring to FIG. 2 , the first light source 130 is disposed between thefirst camera module 110 and the second camera module 120.

FIG. 3 shows an example of a configuration of the second optical guidemodule 120A of the second camera module 120 of FIG. 1 or 2 .

Referring to FIG. 3 , the second optical guide module 120A may include ahyperspectral filter 370, a low pass filter 360, and a lens system 350sequentially stacked on the second image sensor 120B and may furtherinclude other optical members. The hyperspectral filter 370 may includea plurality of pixels for hyperspectral filtering, and each pixel mayinclude a plurality of channels. The low pass filter 360 passes awavelength shorter than a specific wavelength and blocks a wavelengthlonger than the specific wavelength. The low pass filter 360 may be, forexample, a filter that blocks near infrared (NIR). As light 3L3 incidentfrom the subject 150 passes through the hyperspectral filter 370, ahyperspectral image of the subject 150 is recorded in the second imagesensor 120B. The lens system 350 collects the light 3L3 incident fromthe subject 150. The lens system 350 may include a single lens or aplurality of lenses.

FIG. 4 shows an example of the first image sensor 110B of the RGB cameramodule 110 shown in FIG. 1 or 2 . The first image sensor 110B mayinclude a plurality of pixels (e.g., pixels P1 to P4). As shown in FIG.4 , each pixel may include a plurality of sub-pixels. FIG. 5 shows anexample of the hyperspectral filter 370 of FIG. 3 . The hyperspectralfilter 370 includes a plurality of pixels 510, 520, 530, and 540.

Referring to FIGS. 4 and 5 , the four pixels P1 to P4 of the first imagesensor 110B of the RGB camera module 110 may together correspond to onepixel 510 of the hyperspectral filter 370 of the hyperspectral cameramodule 120, which is one of several examples. One pixel of thehyperspectral filter 370 of the hyperspectral camera module 120 maycorrespond to a plurality of pixels of the first image sensor 110B ofthe RGB camera module 110, and as an example, as described above, thefour pixels P1 to P4 of the first image sensor 110B of the RGB cameramodule 110 may correspond to one pixel 510 of the hyperspectral filter370 of the hyperspectral camera module 120. In another example, three,six, eight, or ten pixels of the RGB camera module 110 may correspond toone pixel of the hyperspectral camera module 120. One pixel 510 of thehyperspectral filter 370 of the hyperspectral camera module 120 includesa plurality of channels 1 to 16, which is merely an example. Each of aplurality of pixels 510, 520, 530, and 540 included in the hyperspectralfilter 370 may include 10 or more channels, and may include, forexample, more than several tens to hundreds of channels. Light that maypass through the plurality of channels 1 to 16 included in one pixel 510of the hyperspectral filter 370 may be different from each other. Thatis, each channel shows relatively large transmittance only for light ofa specific wavelength (or light of a specific wavelength band), andthus, light that may pass through each channel may be specified, andlight that may pass through each channel may be different from eachother. As a result, one pixel 510 of the hyperspectral filter 370 maydivide incident light into light having different wavelengths accordingto the number of channels included in the pixel. In other words, lightincident on one pixel 510 of the hyperspectral filter 370 may be dividedby the number of channels of the pixel.

Each pixel (e.g., pixel P1) included in the first image sensor 110B ofthe RGB camera module 110 includes four sub-pixels (one R, one B, andtwo Gs), but it may be said that each pixel of an RGB camera includesthree channels in that three sub-pixels among the four sub-pixelsrespectively transmit only light of a specific wavelength different fromeach other. In other words, light incident on each pixel of the RGBcamera is divided into light having three different wavelengths. In thisregard, it may be said that each pixel of the RGB camera includes threespectral channels. As described below, each pixel of the RGB cameramodule 110 may include six sub-pixels, and, in this case, each pixelincludes six spectral channels.

For convenience of explanation, it is regarded that the four pixels P1to P4 of the first image sensor 110B of the RGB camera module 110together correspond to one pixel 510 of the hyperspectral filter 370 ofthe hyperspectral camera module 120 and one pixel 510 of thehyperspectral filter 370 includes 16 channels 1 to 16. Each of the 16channels 1 to 16 may be viewed as a sub-pixel performing a functionequivalent to a sub-pixel of a unit pixel (for example, P1) of the firstimage sensor 110B of the RGB camera module 110. A total area of thefirst to fourth pixels P1 to P4 of the first image sensor 110B of theRGB camera module 110 may correspond to an area of one pixel 510 of thehyperspectral filter 370. According to this correspondence, as shown inFIG. 6 , one pixel 510 of the hyperspectral filter 370 may be dividedinto four virtual spectral pixels SP1 to SP4. In other words, it may beregarded that one pixel 510 of the hyperspectral filter 370 includes thefirst to fourth spectral pixels SP1 to SP4. The first to fourth spectralpixels SP1 to SP4 may correspond to the first to fourth pixels P1 to P4of the RGB camera module 110. As a result, the first pixel P1 of the RGBcamera module 110 may correspond to the first spectral pixel SP1, thatis, first, second, fifth, and sixth channels 1, 2, 5, and 6 of one pixel510 of the hyperspectral filter 370 as shown in FIG. 6 . The secondpixel P2 of the first image sensor 110B of the RGB camera module 110 maycorrespond to the second spectral pixel SP2, that is, third, fourth,seventh, and eighth channels 3, 4, 7, and 8 of one pixel 510 of thehyperspectral filter 370. The third pixel P3 of the first image sensor100B of the RGB camera module 110 may correspond to the third spectralpixel SP3, that is, ninth, tenth, thirteenth, and fourteenth channels 9,10, 13, and 14 of one pixel 510 of the hyperspectral filter 370. Thefourth pixel P4 of the first image sensor 110B of the RGB camera module110 may correspond to the fourth spectral pixel SP4, that is, eleventh,twelfth, fifteenth, and sixteenth channels 11, 12, 15, and 16 of onepixel 510 of the hyperspectral filter 370. The correspondence may varydepending on the numbers assigned to the channels 1 to 16 of one pixel510 of the hyperspectral filter 370. The above-described correspondencemay be extended to all pixels of the RGB camera module 110 and allpixels included in the hyperspectral filter 370 of the hyperspectralcamera module 120.

As described above, the first to fourth pixels P1 to P4 of the RGBcamera module 110 and the first to fourth spectral pixels SP1 to SP4 ofthe hyperspectral filter 370 may correspond to each other in position orarea, and although not reaching the spectral characteristic of thehyperspectral filter 370, the first to fourth pixels P1 to P4 of the RGBcamera module 110 may also be regarded as having a spectralcharacteristic in a narrow sense, and thus, information obtained fromspectral characteristics of an image acquired through the first tofourth pixels P1 to P4 of the RGB camera module 110 may be used forincreasing the resolution of a hyperspectral image obtained through thehyperspectral camera 120, which will be described later.

FIG. 7 shows an example of intensity distributions of R, G, and B of thefirst to fourth pixels P1 to P4 of the RGB camera module 110 for anexample image.

Referring to FIG. 7 , the R, G, and B intensity distributions of thefirst to fourth pixels P1 to P4 are different from each other. Based onthe intensity distributions, R, G, and B average values (hereinafter,referred to as first average values) of all of the first to fourthpixels P1 to P4 may be obtained, and a difference between the firstaverage value and R, G, and B of each of the first to fourth pixels P1to P4, that is, a deviation may be obtained. Specifically, R values foreach of the first to fourth pixels P1 to P4 may be averaged to obtain afirst average R value, G values for each of the first to fourth pixelsP1 to P4 may be averaged to obtain a first average G value, and B valuesfor each of the first to fourth pixels P1 to P4 may be averaged toobtain a first average B value. First deviations Δ1 between therespective first average R, G, and B values and R, G, and B values ofthe first pixel P1 may be obtained, and second deviations Δ2 between therespective first average R, G, and B values and respective R, G, and Bvalues of the second pixel P2 may be obtained. Also, third deviations Δ3between the respective first average R, G, and B values and respectiveR, G, and B values of the third pixel P3 may be obtained, and fourthdeviations Δ4 between the respective first average R, G, and B valuesand R, G, and B values of the fourth pixel P4 may be obtained. Asdescribed below, the first to fourth deviations Δ1, Δ2, Δ3, and Δ4 maybe used to correct a hyperspectral image.

FIG. 8 shows an example of a hyperspectral spectrum with respect to thesame example image having the R, G, and B distributions of FIG. 7 .Referring to FIG. 8 , it may be seen that the hyperspectral spectrumspans the entire visible light band.

A hyperspectral image based on the hyperspectral spectrum of FIG. 8 maybe obtained through the first to fourth virtual spectral pixels SP1 toSP4. FIG. 9 shows deviations of R, G and B values with respect to thefirst to fourth spectral pixels SP1 to SP4 when the hyperspectral imageis obtained.

R, G, and B values of each of the first to fourth spectral pixels SP1 toSP4 may be obtained based on the obtained hyperspectral image.Accordingly, R, G, and B average values (second average values) withrespect to all of the first to fourth spectral pixels SP1 to SP4 may beobtained. Specifically, R values for each of the first to fourthspectral pixels SP1 to SP4 may be averaged to obtain a second average Rvalue, G values for each of the first to fourth spectral pixels SP1 toSP4 may be averaged to obtain a second average G value, and B values foreach of the first to fourth spectral pixels SP1 to SP4 may be averagedto obtain a second average B value. The second average value may becomethe R, G, and B average values of the first pixel of a hyperspectralfilter. The second average value may include an R average value, a Gaverage value, and a B average value.

In FIG. 9 , a first section 9L1 representing the variation in blue lightB represents a variation range of the B value, that is, a B valuedeviation range of the first to fourth spectral pixels SP1 to SP4 withthe first average B value as the center. A second section 9L2representing the variation in green light G represents the variationrange of the G value, that is, a G value deviation range of the first tofourth spectral pixels SP1 to SP4 with the first average G value as thecenter. A third section 9L3 represents the variation range of the Rvalue, that is, an R value deviation range of the first to fourthspectral pixels SP1 to SP4 with the first average R value as the center.In FIG. 9 , reference numeral 9L4 represents the first average R, G, andB values.

Correction of an image obtained through the first to fourth spectralpixels SP1 to SP4, that is, correction of a hyperspectral image obtainedbased on the hyperspectral spectrum of FIG. 8 may be performed asfollows.

For example, a sum of the second average values and the first deviationsΔ1 mentioned in the description of FIG. 7 may be calculated as correctedR, G, and B values of the first spectral pixel SP1. Also, a sum of thesecond average values and the second deviations Δ2 may be calculated ascorrected R, G, and B values of the second spectral pixel SP2. Also, asum of the second average values and the third deviations Δ3 may becalculated as corrected R, G, and B values of the third spectral pixelSP3, and a sum of the second average values and the fourth deviations Δ4may be calculated as corrected R, G, and B values of the fourth spectralpixel SP4. In this way, corrected R, G, and B values for the first tofourth spectral pixels SP1 to SP4 may be obtained. The corrected R, G,and B values with respect to the first to fourth spectral pixels SP1 toSP4 thus obtained may be, as a result, a corrected image of ahyperspectral image obtained through the first to fourth spectral pixelsSP1 to SP4, that is, a corrected image with respect to the hyperspectralimage based on the hyperspectral spectrum of FIG. 8 .

The configuration of a sub-pixel of each pixel included in the firstimage sensor 110B of the RGB camera module 110 may vary. For example, asillustrated in FIG. 10 , a unit pixel 1000 included in the first imagesensor 110B may be configured of first to fourth sub-pixels R, G, B, andIR different from each other. The first to third sub-pixels R, G, and Bmay be sub-pixels having relatively high transmittances for red light,green light, and blue light, respectively, and the fourth sub-pixel IRmay be a sub-pixel having a relatively high transmittance for infraredlight. Therefore, an image that uses red light, green light, blue light,and infrared light as main light may be obtained through the first imagesensor 110B. In the case when the sub-pixel configuration of a unitpixel of the first image sensor 110B is to be used to detect infraredrays, since the hyperspectral camera module 120 should also beconfigured to receive infrared rays, the low pass filter 360 may beomitted from the hyperspectral camera module 120.

As another example, as illustrated in FIG. 11 , the unit pixel 1100included in the first image sensor 110B may include first to fourthsub-pixels R, G, B, and UV different from those of FIG. 10 . The firstto third sub-pixels R, G, and B of FIG. 11 may be the same as those ofFIG. 10 . The fourth sub-pixel UV of FIG. 11 may be a sub-pixel having arelatively high transmittance to UV light. In the case of FIG. 11 , animage that uses red light, green light, blue light, and UV light as mainlight may be obtained through the first image sensor 110B.

As another example, as illustrated in FIG. 12 , a unit pixel 1200included in the first image sensor 110B includes six sub-pixels, thatis, first to sixth sub-pixels UV, R, G1, IR, G2, and B. The firstsub-pixel UV is a sub-pixel having a relatively high transmittance withrespect to UV light, the second sub-pixel R is a sub-pixel having arelatively high transmittance with respect to red light, and the thirdand fifth sub-pixels G1 and G2 are sub-pixels having a relatively hightransmittance with respect to green light. The third and fifthsub-pixels G1 and G2 may be the same sub-pixels in terms of materialsand/or optical properties. Dividing the sub-pixels for the green lightinto the third and fifth sub-pixels G1 and G2 is for convenience ofexplanation. The fourth sub-pixel IR is a sub-pixel having a relativelyhigh transmittance with respect to infrared light. As a result, when theunit pixel of the first image sensor 110B of the RGB camera module 110is the unit pixel 1200 of FIG. 12 , an image obtained by the RGB cameramodule 110 is formed by using red light, green light, blue light,infrared light, and UV light as main light.

When the first image sensor 110B of the RGB camera module 110 of thedual camera module 100 according to an example embodiment includes theunit pixel 1200 illustrated in FIG. 12 , the application field of thedual camera module 100 may be extended to a visible light band, aninfrared band, and an UV band. For this purpose, infrared light shouldbe incident on the hyperspectral filter 370 of the hyperspectral cameramodule 120, and thus, the low pass filter 360 disposed between thehyperspectral filter 370 and the lens system 350 may be omitted.

As described above, the application field of the dual camera modules 100and 200 may be extended to outside the visible light band by varying theconfiguration of the unit pixels of the first image sensor 110B of theRGB camera module 110. In this case, one of the channels 1 to 16included in the unit pixel 510 of the hyperspectral filter 370 may be aninfrared channel or a UV channel. In another example, the unit pixel 510of the hyperspectral filter 370 may include a separate channel to beused as an infrared channel and/or a separate channel to be used as a UVchannel.

FIG. 13 shows a virtual spectral pixel corresponding to the unit pixels510 of the hyperspectral filter 370 when the unit pixels 510 of thehyperspectral filter 370 of the dual camera modules 100 and 200according to an example embodiment include N channels arranged in an n×marray and a unit pixel of the first image sensor 110B of the RGB cameramodule 110 includes four sub-pixels. In FIG. 13 , N>m>L and L, m, and nare positive integers greater than two.

In FIG. 13 , the left side shows a case in which the unit pixel 510 ofthe hyperspectral filter 370 includes N channels, and the right sideshows virtual spectral pixels arranged in a K×L array corresponding tothe N channels.

As shown in FIG. 13 , the number of spectral pixels corresponding to theunit pixel 510 including N channels is N/4. Accordingly, the unit pixel510 including the N channels of the hyperspectral filter 370 maycorrespond to N/4 among the plurality of pixels included in the firstimage sensor 110B of the RGB camera module 110. Since the number N ofchannels included in the unit pixels of the hyperspectral filter 370 isseveral dozen or more, one pixel 510 included in the hyperspectralfilter 370 may correspond to a plurality of pixels among pixels includedin the first image sensor 110B of the RGB camera module 110.Accordingly, the image resolution of the RGB camera module 110 isgreater than the resolution of the hyperspectral image obtained throughthe hyperspectral camera module 120. Therefore, the resolution of ahyperspectral image obtained through the hyperspectral camera module 120may be increased by applying an image processing technique (for example,an image correction technique) of the RGB camera module 110, whichprovides a relatively high resolution, to the hyperspectral cameramodule 120.

FIG. 14 shows a virtual spectral pixel corresponding to the unit pixel510 of the hyperspectral filter 370 when the unit pixel 510 of thehyperspectral filter 370 of the dual camera modules 100 and 200according to an example embodiment includes N channels arranged in ann×m array and a unit pixel of the first image sensor 110B of the RGBcamera module 110 includes six sub-pixels as depicted in FIG. 12 .

Referring to FIG. 14 , when the unit pixels 510 of the hyperspectralfilter 370 include N channels 1 through N, and the unit pixels of thefirst image sensor 110B of the RGB camera module 110 include sixsub-pixels, the number of spectral pixels corresponding to the unitpixels 510 is N/6.

FIG. 15 is a perspective view of an example of the hyperspectral filter370.

Referring to FIG. 15 , the unit pixel 510 of the hyperspectral filter370 includes a plurality of channels, that is, first to twenty-fourthchannels f1 to f24. The plurality of channels f1 to f24 may be aplurality of filter regions. For convenience of description, thehyperspectral filter 370 is illustrated as including the first totwenty-fourth channels f1 to f24, but may include more or fewer than 24channels. Light 5L incident on the hyperspectral filter 370 includes aplurality of light components. In other words, the light 5L includeslight of a plurality of wavelengths. Each of the first to twenty-fourthchannels f1 to f24 of the hyperspectral filter 370 may have a layerstructure through which only a light component having a specificwavelength may pass. Light filtering characteristics of the first totwenty-fourth channels f1 to f24 included in the hyperspectral filter370 may all be different from each other. Therefore, the light 5Lincident on the hyperspectral filter 370 may be divided into lighthaving 24 different wavelengths while passing through the hyperspectralfilter 370. For example, light of first to sixth wavelengths λ1 to λ6may be emitted through the first to sixth channels f1 to f6,respectively; light of a twelfth wavelength λ12 may be emitted throughthe twelfth channel f12; light of an eighteenth wavelength λ18 may beemitted through the eighteenth channel f18, and light of a twenty-fourthwavelength λ24 may be emitted through the twenty-fourth channel f24.Since the incident light 5L is divided into wavelengths by thehyperspectral filter 370 as described above, an image of each wavelengthincluded in the incident light 5L may be recorded in the second imagesensor 120B. That is, a hyperspectral image may be recorded in thesecond image sensor 120B. Since the incident light 5L is light reflectedfrom the subject 150, as a result, a hyperspectral image of the subject150 is recorded in the second image sensor 120B.

FIG. 16 is a cross-sectional view of an example of a configuration ofany one (e.g., the first channel f1) of the first to twenty-fourthchannels f1 to f24 included in the unit pixel 510 of FIG. 15 .

Referring to FIG. 16 , the first channel f1 includes a first reflectivelayer DL1, a resonance layer ML1, and a second reflective layer DL2 thatare sequentially stacked. The first reflective layer DL1, the resonancelayer ML1, and the second reflective layer DL2 may form a resonancecavity. In the resonance layer ML1, a first material layer 630A and asecond material layer 630B are alternately and horizontally arranged,and a resonance mode may vary according to a pitch P1 of the first andsecond material layers 630A and 630B and a distance D1 between the firstmaterial layers 630A. That is, a wavelength at which a resonance occursin the resonance layer ML1 may vary according to the pitch P1 of thefirst and second material layers 630A and 630B and/or the distance D1between the first material layers 630A. A thickness T1 of the resonancelayer ML1 may also affect the resonance. Accordingly, a wavelengthpassing through the first channel f1 may be varied by changing the pitchP1 of the first and second material layers 630A and 630B of theresonance layer ML1, the distance D1 between the first material layers630A, the thickness T1 of the resonance layer ML1, or any combination ofthese parameters. Accordingly, at least one of the parameters (pitch,distance, and thickness) of the layer configuration of the resonancelayers of the first to twenty-fourth channels f1 to f24 included in theunit filter 510 of FIG. 15 may be different from each other.

In FIG. 16 , the first reflective layer DB1 may be a first distributedBragg reflector (DBR) layer. For example, the first reflective layer DL1includes a first layer 610 and a second layer 620 that are sequentiallystacked with different refractive indices. The first and second layers610 and 620 are alternately stacked, for example, three times. Thenumber of alternating stacks of the first and second layers 610 and 620may be more or less than three. The first layer 610 may be, for example,a SiO₂ layer or may include a SiO₂ layer. The second layer 620 may be,for example, a TiO₂ layer or may include a TiO₂ layer.

The second reflective layer DL2 may be a second DBR layer. For example,the second reflective layer DL2 may include the second layer 620 and thefirst layer 610 that have refractive indices different from each otherand are sequentially stacked. The second layer 620 and the first layer610 that are sequentially stacked are alternately stacked, for example,three times. The number of alternating stacks of the second layer 620and the first layer 610 may be more or less than three. Therefore, boththe uppermost layer of the first reflective layer DL1 and the lowermostlayer of the second reflective layer DL2 may become the second layer620. The first material layer 630A of the resonance layer ML1 may be thesame material as the second layer 620 of the first and second reflectivelayers DL1 and DL2. The second material layer 630B may be the samematerial as the first layer 610 of the first and second reflectivelayers DL1 and DL2.

Next, in a dual camera module according to an example embodiment, anexample in which precise spectral information by using a hyperspectralcamera module is acquired and the resolution of a hyperspectral image isincreased by using an image correction signal obtained from an RGBcamera module will be described.

FIG. 17 is a hyperspectral photograph 1700 of a hand photographed byusing the hyperspectral camera module 120 of the dual camera module 100according to an example embodiment. The hyperspectral photograph 1700includes image information together with spectral information (spectrum)of a hand.

FIG. 18 is a graph showing a light absorption spectrum of a hand due tohemoglobin and melanin, which is obtained from a hyperspectralphotograph of a hand taken by using the hyperspectral camera module 120.In FIG. 18 , a first graph group GP1 shows reference spectrums. A secondgraph group GP2 shows spectrums photographed by using the hyperspectralcamera module 120.

When the first and second graph groups GP1 and GP2 are compared, it maybe seen that the spectrums photographed by using the hyperspectralcamera module 120 coincide well with the reference spectrums.

The hyperspectral photograph 1700 of FIG. 17 is a combination of animage and spectral information. In the case of the hyperspectral cameramodule 120, as the number of spectral channels included in the unitpixels of the hyperspectral filter 370 increases, the spectral accuracyor spectral resolution of the hyperspectral photograph 1700 may beimproved, but the size of the hyperspectral photograph 1700 may bereduced. In other words, the image resolution of the hyperspectralphotograph 1700 may be reduced as the spectral accuracy is improved.

A hyperspectral image having a further improved resolution may beobtained by applying spectral information (e.g., RGB average value,deviation, etc.) with respect to an image of a hand, the same subject,photographed by using the RGB camera module 110 to the image of thehyperspectral photograph 1700 of FIG. 17 .

In detail, FIG. 19 is a photograph of a hand taken by using the RGBcamera module 110. That is, FIG. 19 shows an RGB image of a hand.

FIG. 20 is a hyperspectral photograph taken with the hyperspectralcamera module 120 and shows spectral characteristics (light absorptioncharacteristics) of a hand due to hemoglobin.

When FIGS. 19 and 20 are compared, it may be seen that the size of thehyperspectral image taken by using the hyperspectral camera module 120is much smaller than that of the image taken by using the RGB cameramodule 110.

FIG. 21 shows a hyperspectral image obtained by simply enlarging thehyperspectral image of FIG. 20 to the level of the RGB image of FIG. 19. It may be seen in FIG. 21 that only the spectral information isenlarged as it is and the resolution is not improved.

FIG. 22 shows an image obtained by correcting the hyperspectral image ofFIG. 21 by applying information obtained from the spectralcharacteristics of the image taken by using the RGB camera module 110 tothe hyperspectral image of FIG. 21 . When the hyperspectral image ofFIG. 22 after correction is compared with the hyperspectral image ofFIG. 21 before correction, it may be seen that the resolution of thehyperspectral image after correction is much higher than beforecorrection while the spectral characteristics are generally maintainedin a correction process.

The process of correcting the hyperspectral image of FIG. 21 to thehyperspectral image of FIG. 22 may be performed as follows. For example,if one pixel 510 of the hyperspectral filter 370 of FIG. 6 is used toobtain the hyperspectral image of FIG. 21 , in other words, the first tofourth spectral pixels SP1 to SP4 of FIG. 6 are used, as described indetail in the descriptions of FIGS. 7 and 9 , R, G, and B average valueswith respect to all of the first to fourth spectral pixels SP1 to SP4,that is, the second average values, are obtained, and afterwards, R, G,and B average values, that is, the first average values with respect toall of the first to fourth pixels P1 to P4 of the first image sensor110B of the RGB camera module 110 corresponding to the first to fourthspectral pixels SP1 to SP4 are obtained, and next, the deviationsbetween the first average value and each of the first to fourth pixelsP1 to P4, that is, the first to fourth deviations Δ1, Δ2, Δ3, and Δ4,are obtained.

Next, as shown in FIG. 23 , R, G, and B values of the first spectralpixel SP1 are corrected by adding a first deviation Δ1 to the secondaverage value, R, G, and B values of the second spectral pixel SP2 arecorrected by adding a second deviation Δ2 to the second average value,R, G, B values of a third spectral pixel SP3 are corrected by adding athird deviation Δ3 to the second average value, and R, G, and B valuesof the fourth spectral pixel SP4 are corrected by adding a fourthdeviation Δ4 to the second average value.

Through the corrections, the hyperspectral image of FIG. 21 may become ahyperspectral image with much improved resolution, as shown in FIG. 22 .

As a result, when the dual camera modules 100 and 200 including thehyperspectral camera according to an example embodiment are used, ahyperspectral image having a high resolution may be obtained whilemaintaining superior spectral characteristics (spectrum) of thehyperspectral camera.

The RGB camera module 110 and the hyperspectral camera module 120 arespatially separated from each other in the dual camera modules 100 and200 according to an example embodiment. Next, the effects and advantagesobtained by separation of the camera modules 110 and 120 will bedescribed.

FIG. 24 is a plan view of a case in which a subject 2410 is photographedby using an apparatus 2400 on which a dual camera module according to anexample embodiment is mounted.

Referring to FIG. 24 , the apparatus 2400 may be an example of a mobiledevice, such as a mobile phone. The apparatus 2400 includes first andsecond cameras 2420 and 2430 provided side by side. A light source 2440is built in the left side of the first camera 2420. The light source2440 may be a light source mounted in a general mobile phone. The firstcamera 2420 may be an RGB camera module 110 of the dual camera module100 or 200, or a member including the RGB camera module 110. The secondcamera 2430 may be a hyperspectral camera module 120 of the dual cameramodule 100 or 200 according to an example embodiment or a memberincluding the hyperspectral camera module 120. Among light L11 emittedfrom the light source 2440, light 2L1 directed toward the subject 2410is irradiated onto a front surface of the subject 2410. The light 2L1irradiated on the front surface of the subject 2410 is reflected andincident on the first camera 2420 and the second camera 2430. Referencenumeral 2L2 denotes light reflected from the subject 2410 and incidenton the first camera 2420 and reference numeral 2L3 denotes lightreflected from the subject 2410 and incident on the second camera 2430.The first and second cameras 2420 and 2430 are in a lateral directionarranged side by side at a given interval. Accordingly, a viewing angleof the first camera 2420 viewing the subject 2410 may be different froma viewing angle of the second camera 2430 viewing the subject 2410. Forexample, the first camera 2420 may see a front center of the subject2410, and the second camera 2430 may see a portion that is adjacent tothe front center and is to the right of the front center of the subject2410. That is, a center of the field of view of the second camera 2430is to the right of the front center of the subject 2410 and may be closeto the front center of the subject 2410. Although a viewing angledifference between the first and second cameras 2420 and 2430 viewingthe subject 2410 may not be large in that the interval between the firstand second cameras 2420 and 2430 is not large, due to the viewing angledifference, light 2L3 reflected from the subject 2410 and incident onthe second camera 2430 may not be uniform. Also, since the subject 2410is a three-dimensional object, the degree of non-uniformity of the light2L3 reflected from the subject 2410 and incident on the second camera2430 may be even greater due to the subject 2410 having a curvature andthe light source 2440 being biased in one side. In this case, in thecase of a hyperspectral image observed by the second camera 2430, asshown in an image (a) of FIG. 25 , there may be portions where thespectral characteristics are not uniform. The non-uniform portions maybe difficult to overcome with information by one camera, but it may beconfirmed how much the light source 2440 is biased and/or how much theviewing angle of the second camera 2430 is biased through informationobtained by the two cameras, the first and second cameras 2420 and 2430.An image (b) of FIG. 25 shows a hyperspectral image with improveduniformity. The degree of curvature of the subject 2410 may also beconfirmed from combined information of an RGB image obtained from thefirst camera 2420 and a hyperspectral image obtained from the secondcamera 2430. An accurate hyperspectral image result may be providedthrough the corrected information.

A dual camera module according to an example embodiment may be appliedto security using facial recognition.

In other words, facial recognition may be an important issue insecurity, but existing analysis methods may have limitation due toforgery through a photo. A hyperspectral camera may provide spectralinformation at the same time as information regarding the shape of aface. Accordingly, when using a hyperspectral camera, the accuracy offacial recognition may be increased. In addition to shape recognitionand spectral analysis by using a hyperspectral camera, the recognitionof three-dimensional curved surfaces may be increased by using an RGBcamera and a hyperspectral camera, thereby further improving theaccuracy of facial recognition.

Next, an electronic apparatus including a dual camera module accordingto an example embodiment will be described.

FIG. 26 shows a mobile phone 900 as one of electronic apparatusesincluding a dual camera module according to an example embodiment.

Referring to FIG. 26 , a dual camera module 910 is mounted on a rearsurface 900B of the mobile phone 900. The dual camera module 910includes first and second camera modules 910A and 910B. A light source970 is provided on the right side of the second camera module 910B. Thelight source 970 may be provided on the left side of the first cameramodule 910A or between the first camera module 910A and the secondcamera module 910B. In addition to the light source 970, a second lightsource may further be provided. The first camera module 910A may be thefirst camera module 110 of FIG. 1 or may include the first camera module110. The second camera module 910B may be the second camera module 120of FIG. 1 or may include the second camera module 120. The dual cameramodule 910 may correspond to the dual camera module 100 of FIG. 1 . Thelight source 970 may be the first light source 130 of FIG. 1 . Powerrequired for operating the dual camera module 910 may be supplied from abattery 90B embedded in the mobile phone 900. The mobile phone 900 mayinclude a circuit unit 90C for operation and control thereof. Operationand control of the dual camera module 910 may also be performed throughthe circuit unit 90C. Light 26L2 reflected from a subject 930 may benatural light reflected from the subject 930 and/or light 26L1 that isirradiated to the subject 930 from the light source 970 and then isreflected from the subject 930.

The left figure shows a front side of the mobile phone 900. The frontsurface includes a display region 960. A corrected hyperspectral image930A of the subject 930 is displayed on the display region 960. Thecorrected hyperspectral image 930A is obtained via the dual cameramodule 910. In the display region 960, spectral information with respectto the corrected hyperspectral image 930A may be displayed together withthe corrected hyperspectral image 930A. A hyperspectral image of thesubject 930 before correction may be displayed on the display region 960by operating a function button 980 disposed on the front side. Variousfunctions of the mobile phone 900 may be called or be performed throughthe function button 980.

FIGS. 27 and 28 show a mirror-type display apparatus 2700 (i.e., amirror display apparatus) including a dual camera module according to anexample embodiment. FIG. 27 is a side view and FIG. 28 is a front view.

Referring to FIG. 27 , a subject 2720 may see itself through themirror-type display apparatus 2700, and an image of the subject 2720(e.g., a hyperspectral image of the subject) may be photographed throughthe first and second cameras 2710 and 2712 mounted on the displayapparatus 2700. The display apparatus 2700 may include a driving controlcircuit unit 2716 that performs operations and controls related tophotographing the subject 2720 and operations and controls related todisplaying a hyperspectral image of a photographed subject. The drivingcontrol circuit unit 2716 may be built in the display apparatus 2700 sothat the driving control circuit unit 2716 is not visible from theoutside.

Referring to FIG. 28 , the mirror-type display apparatus 2700 mayinclude first and second cameras 2710 and 2712 on a front surface 2718thereof. The first and second cameras 2710 and 2712 respectively may bedisposed at both sides of the upper end of the front surface 2718thereof. The first camera 2710 may be disposed on the left side and thesecond camera 2712 may be disposed on the right side. The disposition ofthe first and second cameras 2710 and 2712 may be interchanged.Positions of the first and second cameras 2710 and 2712 may be changed.For example, the first and second cameras 2710 and 2712 may bepositioned at the top center of the front surface 2718, or may bedisposed up and down side by side at the top left or right side. One ofthe first and second cameras 2710 and 2712 may be the RGB camera module110 of FIG. 1 or may include the RGB camera module 110, and the otherone may be the hyperspectral camera module 120 of FIG. 1 or may includethe hyperspectral camera module 120. A display region 2760 is located atthe center of the front surface 2718 below the first and second cameras2710 and 2712. The display region 2760 may be a region where an imageand a hyperspectral image photographed with a camera module includingthe first and second cameras 2710 and 2712 or a spectral information,such as spectrum is displayed. When the display apparatus 2700 is in anoff state, the display region 2760 may be used as a general mirror. Thedriving control circuit unit 2716 of FIG. 27 may be disposed behind thedisplay region 2760. Also, if the sizes of the first and second cameras2710 and 2712 are small enough to be inconspicuous, the first and secondcameras 2710 and 2712 may be installed at positions in the displayregion 2760 where the subject 2720 is optimally photographed. Thedisplay region 2760 may be surrounded by a light source 2730. The lightsource 2730 may be a boundary of the display region 2760. The lightsource 2730 may be a boundary that separates the display region 2760from a non-display region, but in another example, the front surface2718 may entirely be a mirror-type display region. In other words, aregion outside the light source 2730 may also be used as a displayregion having the same function as the display area 2760. In anotherexample, the region outside the light source 2730 may be a passivedisplay region, such as a simple mirror rather than an active displayregion, such as the display region 2760. The light source 2730 may havea configuration to emit light in a visible band, or may have aconfiguration to emit infrared rays. The display region 2760 may be acircle or an ellipse, or a shape like a circle or an ellipse, but may bedesigned in a quadrangle or other shape. As a hyperspectral image 2720Aof the subject 2720 is displayed on the display region 2760, informationobtained through the hyperspectral image 2720A or information that maybe obtained through the hyperspectral image 2720A may be displayedtogether. The information may be displayed in a predetermined region2750 in the display region 2760. The information displayed on thepredetermined region 2750 may include information that may be helpfulfor a user's beauty or treatment (e.g., product information for beautyor treatment).

A control unit or an operation unit 2740 is provided in a lower end ofthe front surface 2718. The operation unit 2740 may be a portioncontrolling a turn on/off operation of the display region 2760 or anoperation in the display region 2760. As a result, an overall drivingcontrol of the display apparatus 2700 may be performed through theoperation unit 2740.

Next, a method of operating an electronic apparatus including a dualcamera module according to an example embodiment will be described withreference to FIG. 29 . The method of operation consequently includes amethod of increasing the resolution of a hyperspectral image, and may beperformed as described with reference to FIGS. 7 to 9 based on thespectral pixel concept described with reference to FIG. 6 .

Referring to FIG. 29 , first, in a process of acquiring an RGB image ofthe subject 150 by using the RGB camera module 110, RGB average values(first average values) with respect to pixels of the first image sensor110B of the RGB camera module 110 are obtained (S1), and deviationsbetween the first average values and each of the pixels are obtained(S2). Next, in a process of acquiring the hyperspectral image of thesubject 150 by using the hyperspectral camera module 120, RGB averagevalues (second average values) with respect to pixels of thehyperspectral filter 370 corresponding to the pixels of the first imagesensor 110B of the RGB camera module 110 are obtained (S3). The secondaverage values may be obtained by using the method of obtaining the RGBaverage value with respect to the plurality of spectral pixels aftermapping channels included in the unit pixels of the hyperspectral filter370 to a plurality of virtual spectral pixels as described withreference to FIG. 6 . When the number of sub-pixels constituting theunit pixels included in the first image sensor 110B of the RGB cameramodule 110 is four and the unit pixels included in the hyperspectralfilter 370 have N channels, the number of spectral pixels is N/4, andthe number of the spectral pixels is N/6 when the number of thesub-pixels constituting the unit pixels included in the first imagesensor 110B of the RGB camera module 110 is six. Next, in order toobtain a corrected RGB value for each of the spectral pixels, adeviation of each pixel corresponding to each of the spectral pixels isadded to the second average value (S4). For example, if the firstspectral pixel (SP1 of FIG. 6 ) of the plurality of spectral pixelscorresponds to the first pixel (for example, P1 of FIG. 4 ) of the RGBcamera module 110, a value obtained by adding a deviation (first averagevalue—RGB value of the first pixel P1) with respect to the first pixelP1 to the second average value may be a corrected RGB value with respectto the first spectral pixel SP1. In this way, the correction for theplurality of spectral pixels is completed. By performing this processfor all pixels included in the hyperspectral filter 370, as a result,resolution correction with respect to a hyperspectral image may beachieved.

The dual camera module described above includes an RGB camera module anda hyperspectral camera module, and thus, the resolution of ahyperspectral image obtained by the hyperspectral camera may beincreased by applying spectral information obtained from the RGB cameramodule to the hyperspectral camera module.

As a result, when the dual camera module according to an exampleembodiment is used, a hyperspectral image with increased resolution maybe provided together with accurate spectral information, and thus, afurther accurate face recognition may be achieved in a security field.Also, an error caused by a distance difference between two cameras maybe corrected by using image information obtained from two differentcameras. Also, it is possible to correct an image non-uniformity thatoccurs due to an uneven illumination, a visual angle difference betweenthe two cameras, and/or a surface curvature of a three-dimensionalobject.

It should be understood that example embodiments described herein shouldbe considered in a descriptive sense only and not for purposes oflimitation. Descriptions of features or aspects within each exampleembodiment should typically be considered as available for other similarfeatures or aspects in other embodiments. While example embodiments havebeen described with reference to the figures, it will be understood bythose of ordinary skill in the art that various changes in form anddetails may be made therein without departing from the spirit and scopeas defined by the following claims.

What is claimed is:
 1. An apparatus comprising: a first image sensorconfigured to sense a first image of a subject; and a second imagesensor configured to sense a second image of the subject, wherein eachof the first and second images comprises a spectral image different fromeach other, one of the first and second images comprises an image thatis used to obtain a correction value applied to correction of the otherof the first and second images, and the first and second image sensorsare included in a same device, wherein the other one of the first andsecond images comprises an uncorrected hyperspectral image.
 2. Anapparatus comprising: a first image sensor including first pixels toreceive red light, green light and blue light, the first pixels beingconfigured to provide an image of a subject; a spectral filterconfigured to pass light from the subject; and a second image sensorincluding second pixels to receive the light passing from the spectralfilter, the second pixels being configured to provide a spectral imageof the subject, wherein one second pixel of the second pixels of thesecond image sensor corresponds to a plurality of the first pixels ofthe first image sensor, and wherein the spectral filter comprises afirst reflector, a second reflector, and a resonance layer between thefirst reflector and the second reflector.
 3. The apparatus of claim 2,wherein the spectral image provided by the second image sensor iscorrected based on the image provided by the first image sensor.
 4. Theapparatus of claim 3, wherein the spectral image provided by the secondimage sensor is corrected to increase a resolution of the second imagesensor.
 5. The apparatus of claim 2, wherein one first pixel of thefirst pixels of the first image sensor comprises 4 or 6 sub-pixels. 6.The apparatus of claim 2, wherein each of second pixels of the secondimage sensor corresponds to a plurality of the first pixels of the firstimage sensor.
 7. The apparatus of claim 6, wherein each of the secondpixels of the second image sensor corresponds to four first pixels ofthe first image sensor.
 8. The apparatus of claim 6, wherein each of thesecond pixels of the second image sensor corresponds to six first pixelsof the first image sensor.
 9. The apparatus of claim 2, wherein thespectral filter comprises a plurality of pixels, and each of theplurality of pixels comprises a plurality of channels, and wherein eachchannel of the plurality of channels comprises a first reflective layerand a second reflective layer on the first reflective layer.
 10. Theapparatus of claim 9, wherein each channel of the plurality of channelscomprises a respective resonance layer between the first reflectivelayer and the second reflective layer.
 11. The apparatus of claim 10,wherein the resonance layer comprises a plurality of first materiallayers alternately and horizontally arranged with a plurality of secondmaterial layers.
 12. The apparatus of claim 11, wherein a pitch of theresonance layer of each channel of the plurality of channels isdifferent from a pitch of the resonance layer of each other channel ofthe plurality of channels.
 13. The apparatus of claim 11, wherein awidth of each first material layer of each channel of the plurality ofchannels is different from a width of each first material layer of eachother channel of the plurality of channels.
 14. The apparatus of claim11, wherein a thickness of the resonance layer of each channel of theplurality of channels is different from a thickness of the resonancelayer of each other channel of the plurality of channels.
 15. Theapparatus of claim 9, wherein the first reflective layer comprises afirst layer and a second layer stacked on the first layer.
 16. Theapparatus of claim 15, wherein a refractive index of the first layer isdifferent from a refractive index of the second layer.
 17. The apparatusof claim 9, wherein the first reflective layer comprises a plurality offirst layers alternately and sequentially stacked with a plurality ofsecond layers.
 18. The apparatus of claim 17, wherein a refractive indexof the plurality of first layers is different from a refractive index ofthe plurality of second layers.
 19. The apparatus of claim 9, whereinthe first reflective layer of each channel of the plurality of channelscomprises a first distributed Bragg reflector layer, and wherein thesecond reflective layer of each channel of the plurality of channelscomprises a second distributed Bragg reflector layer.
 20. The apparatusof claim 2, further comprising a low pass filter on an optical path fromthe subject to the second image sensor.
 21. A mobile phone comprisingthe apparatus of claim
 2. 22. The apparatus of claim 2, wherein thespectral image comprises an uncorrected hyperspectral image, and whereinthe image provided by the first image sensor is used to obtain acorrection value applied to correction of the uncorrected hyperspectralimage.